{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tce3stUlHN0L"
      },
      "source": [
        "##### Copyright 2024 Google LLC."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "tuOe1ymfHZPu"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yeadDkMiISin"
      },
      "source": [
        "# REST API: Tuning Quickstart"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lEXQ3OwKIa-O"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://ai.google.dev/palm_docs/tuning_quickstart_rest\"><img src=\"https://ai.google.dev/static/site-assets/images/docs/notebook-site-button.png\" height=\"32\" width=\"32\" />View on ai.google.dev</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/google/generative-ai-docs/blob/main/site/en/palm_docs/tuning_quickstart_rest.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/google/generative-ai-docs/blob/main/site/en/palm_docs/tuning_quickstart_rest.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://ai.google.dev/palm_docs/tuning_quickstart_rest\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\" />Download notebook</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jp_CKyzxUqx6"
      },
      "source": [
        "In this notebook, you'll learn how to get started with the PaLM API tuning service using curl commands or the Python request API to call the PaLM REST API. Here, you'll learn how to tune the text model behind the PaLM API's text generation service."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sCwzzSQqsNys"
      },
      "source": [
        "**Note**: At this time, tuning is only available for the `text-bison-001` model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PSJSI1n7YNv2"
      },
      "source": [
        "## Setup"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SWxKvwd-MSIV"
      },
      "source": [
        "### Authenticate"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JjS8Zy1ojIgc"
      },
      "source": [
        "The PaLM API lets you tune models on your own data. Since it's your data and\n",
        "your tuned models this needs stricter access controls than API-Keys can provide.\n",
        "\n",
        "Before you can run this tutorial, you'll need to\n",
        "[setup OAuth for your project](oauth_quickstart.ipynb).\n",
        "\n",
        "If you want to run this notebook in Colab start by uploading your\n",
        "`client_secret*.json` file using the \"File > Upload\" option.\n",
        "\n",
        "![Show colab's File > Upload option](https://developers.generativeai.google/tutorials/images/colab_upload.png)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JjS834433jIgc"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "client_secret.json"
          ]
        }
      ],
      "source": [
        "!cp client_secret*.json client_secret.json\n",
        "!ls"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I6zTC-3mJ0-2"
      },
      "source": [
        "This gcloud command turns the `client_secret.json` file into credentials that can be used to authenticate with the service.\n",
        "\n",
        "Important: If you're running this in Colab, **don't just click the link it prints**. That will fail. Follow the instriuctions and copy the `gcloud` command it prints to your local machine and run it there, then paste the output from your local machine back here."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9FUwyB_MJ0-2"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "if 'COLAB_RELEASE_TAG' in os.environ:\n",
        "  # Use `--no-browser` in colab\n",
        "  !gcloud auth application-default login --no-browser --client-id-file client_secret.json --scopes='https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/generative-language.tuning'\n",
        "else:\n",
        "  !gcloud auth application-default login --client-id-file client_secret.json --scopes='https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/generative-language.tuning'"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mpe880lfchkp"
      },
      "source": [
        "## Calling the REST API with CURL"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FaSmQssJaMYV"
      },
      "source": [
        "This section gives example curl statements to call the REST API. You will learn how to create a tuning job, check its status and once complete, make an inference call."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g1cQqdypn-Ga"
      },
      "source": [
        "### Set variables"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jrbohImuJFGW"
      },
      "source": [
        "Set variables for recurring values to use for the rest of the REST API calls. The code is using the Python `os` library to set environment variables which is accessible in all the code cells.\n",
        "\n",
        "This is specific to the Colab notebook environment. The code in the next code cell is equivalent to running the following commands in a bash terminal.\n",
        "\n",
        "```bash\n",
        "export access_token=$(gcloud auth application-default print-access-token)\n",
        "export project_id=my-project-id\n",
        "export base_url=https://generativelanguage.googleapis.com\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5ZUa588km3Lx"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "access_token = !gcloud auth application-default print-access-token\n",
        "access_token = '\\n'.join(access_token)\n",
        "\n",
        "os.environ['access_token'] = access_token\n",
        "os.environ['project_id'] = \"project-id\"\n",
        "os.environ['base_url'] = \"https://generativelanguage.googleapis.com\"\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zKeIDL44l7Q9"
      },
      "source": [
        "### List tuned models\n",
        "\n",
        "Verify your authentication setup by listing the currently available tuned models."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "VAqw2D3vYWqm"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "      \"name\": \"tunedModels/testnumbergenerator-fvitocr834l6\",\n",
            "      \"name\": \"tunedModels/my-display-name-81-9wpmc1m920vq\",\n",
            "      \"displayName\": \"my display name 81\",\n",
            "      \"name\": \"tunedModels/number-generator-model-kctlevca1g3q\",\n",
            "      \"name\": \"tunedModels/my-display-name-81-r9wcuda14lyy\",\n",
            "      \"displayName\": \"my display name 81\",\n",
            "      \"name\": \"tunedModels/number-generator-model-w1eabln5adwp\",\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
            "                                 Dload  Upload   Total   Spent    Left  Speed\n",
            "\r",
            "  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r",
            "  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r",
            "100 17583    0 17583    0     0  51600      0 --:--:-- --:--:-- --:--:-- 51563\n"
          ]
        }
      ],
      "source": [
        "%%bash\n",
        "\n",
        "curl -X GET ${base_url}/v1beta3/tunedModels \\\n",
        "    -H 'Content-Type: application/json' \\\n",
        "    -H \"Authorization: Bearer ${access_token}\" \\\n",
        "    -H \"x-goog-user-project: ${project_id}\" | grep name\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t6MN52eRmyjN"
      },
      "source": [
        "### Create tuned model\n",
        "\n",
        "To create a tuned model, you need to pass your dataset to the model in the `training_data` field.\n",
        "\n",
        "For this example, you will tune a model to generate the next number in the sequence. For example, if the input is `1`, the model should output `2`. If the input is `one hundred`, the output should be `one hundred one`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "iBH2NaL6myDN"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{\n",
            "  \"name\": \"tunedModels/number-generator-model-q2d0uism5ivd/operations/xvyx09sjxlmh\",\n",
            "  \"metadata\": {\n",
            "    \"@type\": \"type.googleapis.com/google.ai.generativelanguage.v1beta3.CreateTunedModelMetadata\",\n",
            "    \"totalSteps\": 23,\n",
            "    \"tunedModel\": \"tunedModels/number-generator-model-q2d0uism5ivd\"\n",
            "  }\n",
            "}\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
            "                                 Dload  Upload   Total   Spent    Left  Speed\n",
            "\r",
            "  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r",
            "100  1980    0     0  100  1980      0   1638  0:00:01  0:00:01 --:--:--  1639\r",
            "100  2277    0   297  100  1980    146    975  0:00:02  0:00:02 --:--:--  1121\r",
            "100  2277    0   297  100  1980    146    975  0:00:02  0:00:02 --:--:--  1121\n"
          ]
        }
      ],
      "source": [
        "%%bash\n",
        "\n",
        "curl -X POST ${base_url}/v1beta3/tunedModels \\\n",
        "    -H 'Content-Type: application/json' \\\n",
        "    -H \"Authorization: Bearer ${access_token}\" \\\n",
        "    -H \"x-goog-user-project: ${project_id}\" \\\n",
        "    -d '\n",
        "      {\n",
        "        \"display_name\": \"number generator model\",\n",
        "        \"base_model\": \"models/text-bison-001\",\n",
        "        \"tuning_task\": {\n",
        "          \"hyperparameters\": {\n",
        "            \"batch_size\": 2,\n",
        "            \"learning_rate\": 0.001,\n",
        "            \"epoch_count\":3,\n",
        "          },\n",
        "          \"training_data\": {\n",
        "            \"examples\": {\n",
        "              \"examples\": [\n",
        "                {\n",
        "                    \"text_input\": \"1\",\n",
        "                    \"output\": \"2\",\n",
        "                },{\n",
        "                    \"text_input\": \"3\",\n",
        "                    \"output\": \"4\",\n",
        "                },{\n",
        "                    \"text_input\": \"-3\",\n",
        "                    \"output\": \"-2\",\n",
        "                },{\n",
        "                    \"text_input\": \"twenty two\",\n",
        "                    \"output\": \"twenty three\",\n",
        "                },{\n",
        "                    \"text_input\": \"two hundred\",\n",
        "                    \"output\": \"two hundred one\",\n",
        "                },{\n",
        "                    \"text_input\": \"ninety nine\",\n",
        "                    \"output\": \"one hundred\",\n",
        "                },{\n",
        "                    \"text_input\": \"8\",\n",
        "                    \"output\": \"9\",\n",
        "                },{\n",
        "                    \"text_input\": \"-98\",\n",
        "                    \"output\": \"-97\",\n",
        "                },{\n",
        "                    \"text_input\": \"1,000\",\n",
        "                    \"output\": \"1,001\",\n",
        "                },{\n",
        "                    \"text_input\": \"10,100,000\",\n",
        "                    \"output\": \"10,100,001\",\n",
        "                },{\n",
        "                    \"text_input\": \"thirteen\",\n",
        "                    \"output\": \"fourteen\",\n",
        "                },{\n",
        "                    \"text_input\": \"eighty\",\n",
        "                    \"output\": \"eighty one\",\n",
        "                },{\n",
        "                    \"text_input\": \"one\",\n",
        "                    \"output\": \"two\",\n",
        "                },{\n",
        "                    \"text_input\": \"three\",\n",
        "                    \"output\": \"four\",\n",
        "                },{\n",
        "                    \"text_input\": \"seven\",\n",
        "                    \"output\": \"eight\",\n",
        "                }\n",
        "              ]\n",
        "            }\n",
        "          }\n",
        "        }\n",
        "      }' | tee tunemodel.json\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ad5ZWb_rmAst"
      },
      "source": [
        "### Get tuned model state"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4hre6xAdrRyS"
      },
      "source": [
        "The state of the model is set to `CREATING` during training and will change to `ACTIVE` once its complete.\n",
        "\n",
        "Below is a bit of python code to parse out the generated model name from the response JSON. If you're running this in a terminal you can try using a bash JSON parser to parse the response."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BVs6r1j2YIuv"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "tunedModels/number-generator-model-q2d0uism5ivd\n"
          ]
        }
      ],
      "source": [
        "import json\n",
        "\n",
        "first_page = json.load(open('tunemodel.json'))\n",
        "os.environ['modelname'] = first_page['metadata']['tunedModel']\n",
        "\n",
        "print(os.environ['modelname'])\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I1urOyFnd7_N"
      },
      "source": [
        "  Do another `GET` request with the model name to get the model metadata which includes the state field."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6MdwY7duYmYL"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "  \"state\": \"CREATING\",\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
            "                                 Dload  Upload   Total   Spent    Left  Speed\n",
            "\r",
            "  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r",
            "100   494    0   494    0     0    761      0 --:--:-- --:--:-- --:--:--   761\r",
            "100   494    0   494    0     0    760      0 --:--:-- --:--:-- --:--:--   760\n",
            "curl: (3) URL using bad/illegal format or missing URL\n"
          ]
        }
      ],
      "source": [
        "%%bash\n",
        "\n",
        "curl -X GET ${base_url}/v1beta3/${modelname} \\\n",
        "    -H 'Content-Type: application/json' \\\n",
        "    -H \"Authorization: Bearer ${access_token}\" \\\n",
        "    -H \"x-goog-user-project: ${project_id}\" \\ | grep state\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Qg8_cgelayQ5"
      },
      "source": [
        "### Run inference\n",
        "\n",
        "Once your tuning job is finished, you can use it to generate text with the text service."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "15ndxGP_cNBp"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "      \"output\": \"3 2 1\",\n",
            "      \"output\": \"3 2\",\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n",
            "                                 Dload  Upload   Total   Spent    Left  Speed\n",
            "\r",
            "  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r",
            "100   122    0     0  100   122      0    595 --:--:-- --:--:-- --:--:--   595\r",
            "100   122    0     0  100   122      0    101  0:00:01  0:00:01 --:--:--   101\r",
            "100   122    0     0  100   122      0     55  0:00:02  0:00:02 --:--:--    55\r",
            "100   122    0     0  100   122      0     38  0:00:03  0:00:03 --:--:--    37\r",
            "100   122    0     0  100   122      0     28  0:00:04  0:00:04 --:--:--    28\r",
            "100   122    0     0  100   122      0     23  0:00:05  0:00:05 --:--:--     0\r",
            "100   122    0     0  100   122      0     19  0:00:06  0:00:06 --:--:--     0\r",
            "100   122    0     0  100   122      0     16  0:00:07  0:00:07 --:--:--     0\r",
            "100  1569    0  1447  100   122    183     15  0:00:08  0:00:07  0:00:01   310\n"
          ]
        }
      ],
      "source": [
        "%%bash\n",
        "\n",
        "curl -X POST ${base_url}/v1beta3/${modelname}:generateText \\\n",
        "    -H 'Content-Type: application/json' \\\n",
        "    -H \"Authorization: Bearer ${access_token}\" \\\n",
        "    -H \"x-goog-user-project: ${project_id}\" \\\n",
        "    -d '{\n",
        "        \"prompt\": {\n",
        "              \"text\": \"4\"\n",
        "              },\n",
        "        \"temperature\": 1.0,\n",
        "        \"candidate_count\": 2}' | grep output"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r2zvbGFYeR18"
      },
      "source": [
        "The output from your model may or may not be correct. If the tuned model isn't performing up to your required standards, you can try adding more high quality examples, tweaking the hyperparameters or adding a preamble to your examples. You can even create another tuned model based on the first one you created.\n",
        "\n",
        "See the [tuning guide](../guide/model_tuning_guidance) for more guidance on improving performance."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fHWwC2dqXYUb"
      },
      "source": [
        "## Call the REST API with Python requests\n",
        "\n",
        "You can call the rest API with any library that allows you to send http requests.\n",
        "The next set of examples use the Python requests library, and demonstrates some of the  more advanced features."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jZ0jOtfwaxU9"
      },
      "source": [
        "### Set variables"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "a_QXMiSlav_F"
      },
      "outputs": [],
      "source": [
        "access_token = !gcloud auth application-default print-access-token\n",
        "access_token = '\\n'.join(access_token)\n",
        "\n",
        "project = 'project-id'\n",
        "base_url = \"https://generativelanguage.googleapis.com\"\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZWjmINASa-Tg"
      },
      "source": [
        "Import the `requests` library."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QyjlnDjhYWoe"
      },
      "outputs": [],
      "source": [
        "import requests\n",
        "import json"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CPtF6TiUhRr2"
      },
      "source": [
        "### List tuned models\n",
        "\n",
        "Verify your authentication setup by listing the currently available tuned models."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "r44T32P8ZiH4"
      },
      "outputs": [],
      "source": [
        "headers={\n",
        "  'Authorization': 'Bearer ' + access_token,\n",
        "  'Content-Type': 'application/json',\n",
        "  'x-goog-user-project': project\n",
        "}\n",
        "\n",
        "result = requests.get(\n",
        "  url=f'{base_url}/v1beta3/tunedModels',\n",
        "  headers = headers,\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rC8lDO8uh1PY"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'tunedModels': [{'name': 'tunedModels/testnumbergenerator-fvitocr834l6',\n",
              "   'baseModel': 'models/text-bison-001',\n",
              "   'displayName': 'test_number_generator',\n",
              "   'description': '{\"description\":\"generates the  next number in the sequence given the input text\",\"exampleInput\":\"input: 1\",\"exampleOutput\":\"output: 2\",\"datasourceUrl\":\"https://drive.google.com/open?id=11Pdm6GNom4vlBMUHwO6yFjGQT3t1yi44WVShXMFnkVA&authuser=0&resourcekey=0-2d17tccbdBoThXMkNDvtag\",\"showedTuningComplete\":false}',\n",
              "   'state': 'ACTIVE',\n",
              "   'createTime': '2023-09-18T11:06:39.092786Z',\n",
              "   'updateTime': '2023-09-18T11:07:24.198359Z',\n",
              "   'tuningTask': {'startTime': '2023-09-18T11:06:39.461814784Z',\n",
              "    'completeTime': '2023-09-18T11:07:24.198359Z',\n",
              "    'snapshots': [{'step': 1,\n",
              "      'meanLoss': 16.613504,\n",
              "      'computeTime': '2023-09-18T11:06:44.532937624Z'},\n",
              "     {'step': 2,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 20.299532,\n",
              "      'computeTime': '2023-09-18T11:06:47.825134421Z'},\n",
              "     {'step': 3,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 8.169708,\n",
              "      'computeTime': '2023-09-18T11:06:50.580344344Z'},\n",
              "     {'step': 4,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 3.7588992,\n",
              "      'computeTime': '2023-09-18T11:06:53.219133748Z'},\n",
              "     {'step': 5,\n",
              "      'epoch': 3,\n",
              "      'meanLoss': 2.0643115,\n",
              "      'computeTime': '2023-09-18T11:06:55.828458606Z'},\n",
              "     {'step': 6,\n",
              "      'epoch': 3,\n",
              "      'meanLoss': 1.9765375,\n",
              "      'computeTime': '2023-09-18T11:06:58.426053772Z'},\n",
              "     {'step': 7,\n",
              "      'epoch': 4,\n",
              "      'meanLoss': 0.9276156,\n",
              "      'computeTime': '2023-09-18T11:07:01.231832398Z'},\n",
              "     {'step': 8,\n",
              "      'epoch': 5,\n",
              "      'meanLoss': 1.8424839,\n",
              "      'computeTime': '2023-09-18T11:07:03.822710074Z'},\n",
              "     {'step': 9,\n",
              "      'epoch': 5,\n",
              "      'meanLoss': 1.1747926,\n",
              "      'computeTime': '2023-09-18T11:07:06.441685551Z'},\n",
              "     {'step': 10,\n",
              "      'epoch': 6,\n",
              "      'meanLoss': 0.3079359,\n",
              "      'computeTime': '2023-09-18T11:07:08.793491157Z'},\n",
              "     {'step': 11,\n",
              "      'epoch': 7,\n",
              "      'meanLoss': 0.543368,\n",
              "      'computeTime': '2023-09-18T11:07:11.393264892Z'},\n",
              "     {'step': 12,\n",
              "      'epoch': 7,\n",
              "      'meanLoss': 0.35068464,\n",
              "      'computeTime': '2023-09-18T11:07:13.808021238Z'},\n",
              "     {'step': 13,\n",
              "      'epoch': 8,\n",
              "      'meanLoss': 0.026032856,\n",
              "      'computeTime': '2023-09-18T11:07:16.295972078Z'},\n",
              "     {'step': 14,\n",
              "      'epoch': 8,\n",
              "      'meanLoss': 0.108341046,\n",
              "      'computeTime': '2023-09-18T11:07:18.941247488Z'},\n",
              "     {'step': 15,\n",
              "      'epoch': 9,\n",
              "      'meanLoss': 0.016470395,\n",
              "      'computeTime': '2023-09-18T11:07:21.607654306Z'},\n",
              "     {'step': 16,\n",
              "      'epoch': 10,\n",
              "      'meanLoss': 0.063049875,\n",
              "      'computeTime': '2023-09-18T11:07:24.077271307Z'}],\n",
              "    'hyperparameters': {'epochCount': 10,\n",
              "     'batchSize': 16,\n",
              "     'learningRate': 0.02}},\n",
              "   'temperature': 0.7,\n",
              "   'topP': 0.95,\n",
              "   'topK': 40},\n",
              "  {'name': 'tunedModels/my-display-name-81-9wpmc1m920vq',\n",
              "   'baseModel': 'models/text-bison-tuning-test',\n",
              "   'displayName': 'my display name 81',\n",
              "   'state': 'ACTIVE',\n",
              "   'createTime': '2023-09-18T22:02:08.690991Z',\n",
              "   'updateTime': '2023-09-18T22:02:28.806318Z',\n",
              "   'tuningTask': {'startTime': '2023-09-18T22:02:09.161100369Z',\n",
              "    'completeTime': '2023-09-18T22:02:28.806318Z',\n",
              "    'snapshots': [{'step': 1,\n",
              "      'meanLoss': 7.2774773,\n",
              "      'computeTime': '2023-09-18T22:02:12.453056368Z'},\n",
              "     {'step': 2,\n",
              "      'meanLoss': 6.1902447,\n",
              "      'computeTime': '2023-09-18T22:02:13.789508217Z'},\n",
              "     {'step': 3,\n",
              "      'meanLoss': 5.5545835,\n",
              "      'computeTime': '2023-09-18T22:02:15.136220505Z'},\n",
              "     {'step': 4,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 7.9237704,\n",
              "      'computeTime': '2023-09-18T22:02:16.474358517Z'},\n",
              "     {'step': 5,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 7.6770706,\n",
              "      'computeTime': '2023-09-18T22:02:17.758261108Z'},\n",
              "     {'step': 6,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 7.378622,\n",
              "      'computeTime': '2023-09-18T22:02:19.114072224Z'},\n",
              "     {'step': 7,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 4.485537,\n",
              "      'computeTime': '2023-09-18T22:02:20.927434115Z'},\n",
              "     {'step': 8,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 6.815181,\n",
              "      'computeTime': '2023-09-18T22:02:22.267906011Z'},\n",
              "     {'step': 9,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 6.411363,\n",
              "      'computeTime': '2023-09-18T22:02:24.078114085Z'},\n",
              "     {'step': 10,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 8.585093,\n",
              "      'computeTime': '2023-09-18T22:02:25.441598938Z'},\n",
              "     {'step': 11,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 4.901249,\n",
              "      'computeTime': '2023-09-18T22:02:27.108985392Z'},\n",
              "     {'step': 12,\n",
              "      'epoch': 3,\n",
              "      'meanLoss': 7.073003,\n",
              "      'computeTime': '2023-09-18T22:02:28.441662034Z'}],\n",
              "    'hyperparameters': {'epochCount': 3,\n",
              "     'batchSize': 4,\n",
              "     'learningRate': 0.001}},\n",
              "   'temperature': 0.7,\n",
              "   'topP': 0.95,\n",
              "   'topK': 40},\n",
              "  {'name': 'tunedModels/number-generator-model-kctlevca1g3q',\n",
              "   'baseModel': 'models/text-bison-tuning-test',\n",
              "   'displayName': 'number generator model',\n",
              "   'state': 'ACTIVE',\n",
              "   'createTime': '2023-09-18T23:43:21.461545Z',\n",
              "   'updateTime': '2023-09-18T23:43:49.205493Z',\n",
              "   'tuningTask': {'startTime': '2023-09-18T23:43:21.542403958Z',\n",
              "    'completeTime': '2023-09-18T23:43:49.205493Z',\n",
              "    'snapshots': [{'step': 1,\n",
              "      'meanLoss': 7.342065,\n",
              "      'computeTime': '2023-09-18T23:43:23.356271969Z'},\n",
              "     {'step': 2,\n",
              "      'meanLoss': 7.255807,\n",
              "      'computeTime': '2023-09-18T23:43:24.620248223Z'},\n",
              "     {'step': 3,\n",
              "      'meanLoss': 5.4591417,\n",
              "      'computeTime': '2023-09-18T23:43:25.854505395Z'},\n",
              "     {'step': 4,\n",
              "      'meanLoss': 6.968665,\n",
              "      'computeTime': '2023-09-18T23:43:27.138260198Z'},\n",
              "     {'step': 5,\n",
              "      'meanLoss': 4.578809,\n",
              "      'computeTime': '2023-09-18T23:43:28.404943274Z'},\n",
              "     {'step': 6,\n",
              "      'meanLoss': 6.4862137,\n",
              "      'computeTime': '2023-09-18T23:43:29.631624883Z'},\n",
              "     {'step': 7,\n",
              "      'meanLoss': 9.781939,\n",
              "      'computeTime': '2023-09-18T23:43:30.801341449Z'},\n",
              "     {'step': 8,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 5.990006,\n",
              "      'computeTime': '2023-09-18T23:43:31.854703315Z'},\n",
              "     {'step': 9,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 8.846312,\n",
              "      'computeTime': '2023-09-18T23:43:33.075785103Z'},\n",
              "     {'step': 10,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 6.1585655,\n",
              "      'computeTime': '2023-09-18T23:43:34.310432174Z'},\n",
              "     {'step': 11,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 4.7877502,\n",
              "      'computeTime': '2023-09-18T23:43:35.381582526Z'},\n",
              "     {'step': 12,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 9.660514,\n",
              "      'computeTime': '2023-09-18T23:43:36.445446408Z'},\n",
              "     {'step': 13,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 5.6482882,\n",
              "      'computeTime': '2023-09-18T23:43:37.603237821Z'},\n",
              "     {'step': 14,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 3.162092,\n",
              "      'computeTime': '2023-09-18T23:43:38.671463397Z'},\n",
              "     {'step': 15,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 6.322996,\n",
              "      'computeTime': '2023-09-18T23:43:39.769742201Z'},\n",
              "     {'step': 16,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 6.781,\n",
              "      'computeTime': '2023-09-18T23:43:40.985967994Z'},\n",
              "     {'step': 17,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 5.136773,\n",
              "      'computeTime': '2023-09-18T23:43:42.235469710Z'},\n",
              "     {'step': 18,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 7.2091155,\n",
              "      'computeTime': '2023-09-18T23:43:43.415178581Z'},\n",
              "     {'step': 19,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 7.7508755,\n",
              "      'computeTime': '2023-09-18T23:43:44.775221774Z'},\n",
              "     {'step': 20,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 8.144815,\n",
              "      'computeTime': '2023-09-18T23:43:45.788824334Z'},\n",
              "     {'step': 21,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 5.485137,\n",
              "      'computeTime': '2023-09-18T23:43:46.812663998Z'},\n",
              "     {'step': 22,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 3.709197,\n",
              "      'computeTime': '2023-09-18T23:43:47.971764087Z'},\n",
              "     {'step': 23,\n",
              "      'epoch': 3,\n",
              "      'meanLoss': 6.0069466,\n",
              "      'computeTime': '2023-09-18T23:43:49.004191079Z'}],\n",
              "    'hyperparameters': {'epochCount': 3,\n",
              "     'batchSize': 2,\n",
              "     'learningRate': 0.001}},\n",
              "   'temperature': 0.7,\n",
              "   'topP': 0.95,\n",
              "   'topK': 40},\n",
              "  {'name': 'tunedModels/my-display-name-81-r9wcuda14lyy',\n",
              "   'baseModel': 'models/text-bison-tuning-test',\n",
              "   'displayName': 'my display name 81',\n",
              "   'state': 'ACTIVE',\n",
              "   'createTime': '2023-09-18T23:52:06.980185Z',\n",
              "   'updateTime': '2023-09-18T23:52:26.679601Z',\n",
              "   'tuningTask': {'startTime': '2023-09-18T23:52:07.616953503Z',\n",
              "    'completeTime': '2023-09-18T23:52:26.679601Z',\n",
              "    'snapshots': [{'step': 1,\n",
              "      'meanLoss': 7.2774773,\n",
              "      'computeTime': '2023-09-18T23:52:10.278936662Z'},\n",
              "     {'step': 2,\n",
              "      'meanLoss': 6.2793097,\n",
              "      'computeTime': '2023-09-18T23:52:11.630844790Z'},\n",
              "     {'step': 3,\n",
              "      'meanLoss': 5.540499,\n",
              "      'computeTime': '2023-09-18T23:52:13.027840389Z'},\n",
              "     {'step': 4,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 7.977523,\n",
              "      'computeTime': '2023-09-18T23:52:14.368199020Z'},\n",
              "     {'step': 5,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 7.6197805,\n",
              "      'computeTime': '2023-09-18T23:52:15.872428752Z'},\n",
              "     {'step': 6,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 7.3851357,\n",
              "      'computeTime': '2023-09-18T23:52:17.213094182Z'},\n",
              "     {'step': 7,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 4.5342345,\n",
              "      'computeTime': '2023-09-18T23:52:19.090698421Z'},\n",
              "     {'step': 8,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 6.8603754,\n",
              "      'computeTime': '2023-09-18T23:52:20.494844731Z'},\n",
              "     {'step': 9,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 6.418575,\n",
              "      'computeTime': '2023-09-18T23:52:21.815997555Z'},\n",
              "     {'step': 10,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 8.659064,\n",
              "      'computeTime': '2023-09-18T23:52:23.524287192Z'},\n",
              "     {'step': 11,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 4.856765,\n",
              "      'computeTime': '2023-09-18T23:52:24.864661291Z'},\n",
              "     {'step': 12,\n",
              "      'epoch': 3,\n",
              "      'meanLoss': 7.1078596,\n",
              "      'computeTime': '2023-09-18T23:52:26.225055381Z'}],\n",
              "    'hyperparameters': {'epochCount': 3,\n",
              "     'batchSize': 4,\n",
              "     'learningRate': 0.001}},\n",
              "   'temperature': 0.7,\n",
              "   'topP': 0.95,\n",
              "   'topK': 40},\n",
              "  {'name': 'tunedModels/number-generator-model-w1eabln5adwp',\n",
              "   'baseModel': 'models/text-bison-tuning-test',\n",
              "   'displayName': 'number generator model',\n",
              "   'state': 'ACTIVE',\n",
              "   'createTime': '2023-09-19T19:29:08.622497Z',\n",
              "   'updateTime': '2023-09-19T19:29:46.063853Z',\n",
              "   'tuningTask': {'startTime': '2023-09-19T19:29:08.806930486Z',\n",
              "    'completeTime': '2023-09-19T19:29:46.063853Z',\n",
              "    'snapshots': [{'step': 1,\n",
              "      'meanLoss': 7.342065,\n",
              "      'computeTime': '2023-09-19T19:29:13.023811994Z'},\n",
              "     {'step': 2,\n",
              "      'meanLoss': 7.1960244,\n",
              "      'computeTime': '2023-09-19T19:29:14.844046282Z'},\n",
              "     {'step': 3,\n",
              "      'meanLoss': 5.480289,\n",
              "      'computeTime': '2023-09-19T19:29:16.596884354Z'},\n",
              "     {'step': 4,\n",
              "      'meanLoss': 6.851822,\n",
              "      'computeTime': '2023-09-19T19:29:17.741735378Z'},\n",
              "     {'step': 5,\n",
              "      'meanLoss': 4.5535283,\n",
              "      'computeTime': '2023-09-19T19:29:18.914760812Z'},\n",
              "     {'step': 6,\n",
              "      'meanLoss': 6.449012,\n",
              "      'computeTime': '2023-09-19T19:29:20.053316042Z'},\n",
              "     {'step': 7,\n",
              "      'meanLoss': 9.842458,\n",
              "      'computeTime': '2023-09-19T19:29:21.371286675Z'},\n",
              "     {'step': 8,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 5.9831877,\n",
              "      'computeTime': '2023-09-19T19:29:22.915277044Z'},\n",
              "     {'step': 9,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 8.936815,\n",
              "      'computeTime': '2023-09-19T19:29:24.666461680Z'},\n",
              "     {'step': 10,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 6.14651,\n",
              "      'computeTime': '2023-09-19T19:29:26.793310451Z'},\n",
              "     {'step': 11,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 4.853589,\n",
              "      'computeTime': '2023-09-19T19:29:28.328297535Z'},\n",
              "     {'step': 12,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 9.6831045,\n",
              "      'computeTime': '2023-09-19T19:29:29.501236840Z'},\n",
              "     {'step': 13,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 5.706586,\n",
              "      'computeTime': '2023-09-19T19:29:30.612807978Z'},\n",
              "     {'step': 14,\n",
              "      'epoch': 1,\n",
              "      'meanLoss': 3.276942,\n",
              "      'computeTime': '2023-09-19T19:29:31.928747103Z'},\n",
              "     {'step': 15,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 6.1736736,\n",
              "      'computeTime': '2023-09-19T19:29:33.588699180Z'},\n",
              "     {'step': 16,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 6.857398,\n",
              "      'computeTime': '2023-09-19T19:29:35.239083809Z'},\n",
              "     {'step': 17,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 5.098094,\n",
              "      'computeTime': '2023-09-19T19:29:37.000705047Z'},\n",
              "     {'step': 18,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 7.27724,\n",
              "      'computeTime': '2023-09-19T19:29:38.532313231Z'},\n",
              "     {'step': 19,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 7.6310735,\n",
              "      'computeTime': '2023-09-19T19:29:39.696034301Z'},\n",
              "     {'step': 20,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 8.152623,\n",
              "      'computeTime': '2023-09-19T19:29:40.803342042Z'},\n",
              "     {'step': 21,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 5.451577,\n",
              "      'computeTime': '2023-09-19T19:29:42.445788199Z'},\n",
              "     {'step': 22,\n",
              "      'epoch': 2,\n",
              "      'meanLoss': 3.7990716,\n",
              "      'computeTime': '2023-09-19T19:29:43.866737307Z'},\n",
              "     {'step': 23,\n",
              "      'epoch': 3,\n",
              "      'meanLoss': 6.120624,\n",
              "      'computeTime': '2023-09-19T19:29:45.599248553Z'}],\n",
              "    'hyperparameters': {'epochCount': 3,\n",
              "     'batchSize': 2,\n",
              "     'learningRate': 0.001}},\n",
              "   'temperature': 0.7,\n",
              "   'topP': 0.95,\n",
              "   'topK': 40}]}"
            ]
          },
          "execution_count": 20,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "result.json()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ht_JVuo7hU8n"
      },
      "source": [
        "### Create tuned model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B4KrCiYITzXy"
      },
      "source": [
        "Same as for the Curl example, you pass in the dataset through the `training_data` field."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "hh4gcz6aaDzA"
      },
      "outputs": [],
      "source": [
        "operation = requests.post(\n",
        "    url = f'{base_url}/v1beta3/tunedModels',\n",
        "    headers=headers,\n",
        "    json= {\n",
        "        \"display_name\": \"number generator\",\n",
        "        \"base_model\": \"models/text-bison-001\",\n",
        "        \"tuning_task\": {\n",
        "          \"hyperparameters\": {\n",
        "            \"batch_size\": 4,\n",
        "            \"learning_rate\": 0.001,\n",
        "            \"epoch_count\":3,\n",
        "          },\n",
        "          \"training_data\": {\n",
        "            \"examples\": {\n",
        "              \"examples\": [\n",
        "                {\n",
        "                    'text_input': '1',\n",
        "                    'output': '2',\n",
        "                },{\n",
        "                    'text_input': '3',\n",
        "                    'output': '4',\n",
        "                },{\n",
        "                    'text_input': '-3',\n",
        "                    'output': '-2',\n",
        "                },{\n",
        "                    'text_input': 'twenty two',\n",
        "                    'output': 'twenty three',\n",
        "                },{\n",
        "                    'text_input': 'two hundred',\n",
        "                    'output': 'two hundred one',\n",
        "                },{\n",
        "                    'text_input': 'ninety nine',\n",
        "                    'output': 'one hundred',\n",
        "                },{\n",
        "                    'text_input': '8',\n",
        "                    'output': '9',\n",
        "                },{\n",
        "                    'text_input': '-98',\n",
        "                    'output': '-97',\n",
        "                },{\n",
        "                    'text_input': '1,000',\n",
        "                    'output': '1,001',\n",
        "                },{\n",
        "                    'text_input': '10,100,000',\n",
        "                    'output': '10,100,001',\n",
        "                },{\n",
        "                    'text_input': 'thirteen',\n",
        "                    'output': 'fourteen',\n",
        "                },{\n",
        "                    'text_input': 'eighty',\n",
        "                    'output': 'eighty one',\n",
        "                },{\n",
        "                    'text_input': 'one',\n",
        "                    'output': 'two',\n",
        "                },{\n",
        "                    'text_input': 'three',\n",
        "                    'output': 'four',\n",
        "                },{\n",
        "                    'text_input': 'seven',\n",
        "                    'output': 'eight',\n",
        "                }\n",
        "              ]\n",
        "            }\n",
        "          }\n",
        "        }\n",
        "      }\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6cMp2okEnIKR"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<Response [200]>"
            ]
          },
          "execution_count": 40,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "operation"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "L-scSZQ7aoqG"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'name': 'tunedModels/number-generator-ncqqnysl74dt/operations/qqlbwzfyzn0k',\n",
              " 'metadata': {'@type': 'type.googleapis.com/google.ai.generativelanguage.v1beta3.CreateTunedModelMetadata',\n",
              "  'totalSteps': 12,\n",
              "  'tunedModel': 'tunedModels/number-generator-ncqqnysl74dt'}}"
            ]
          },
          "execution_count": 41,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "operation.json()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2ymEe60etcsu"
      },
      "source": [
        "Set a variable with the name of your tuned model to use for the rest of the calls."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "iGyc1yWOCRMz"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'tunedModels/number-generator-ncqqnysl74dt'"
            ]
          },
          "execution_count": 42,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "name=operation.json()[\"metadata\"][\"tunedModel\"]\n",
        "name\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xq1vdaK4hZEI"
      },
      "source": [
        "### Get tuned model state"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xO0Ms0K5hcWM"
      },
      "source": [
        "You can check the progress of your tuning job by checking the state field. `CREATING` means the tuning job is still ongoing and `ACTIVE` means the trainins is complete and the tuned model is ready to use."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ms6u3X7vel12"
      },
      "outputs": [],
      "source": [
        "tuned_model = requests.get(\n",
        "    url = f'{base_url}/v1beta3/{name}',\n",
        "    headers=headers,\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ohfKsMlYfH7m"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'name': 'tunedModels/number-generator-ncqqnysl74dt',\n",
              " 'baseModel': 'models/text-bison-001',\n",
              " 'displayName': 'number generator',\n",
              " 'state': 'CREATING',\n",
              " 'createTime': '2023-09-19T19:56:25.999303Z',\n",
              " 'updateTime': '2023-09-19T19:56:25.999303Z',\n",
              " 'tuningTask': {'startTime': '2023-09-19T19:56:26.297862545Z',\n",
              "  'hyperparameters': {'epochCount': 3, 'batchSize': 4, 'learningRate': 0.001}},\n",
              " 'temperature': 0.7,\n",
              " 'topP': 0.95,\n",
              " 'topK': 40}"
            ]
          },
          "execution_count": 44,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "tuned_model.json()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WEbNjFRKaFbX"
      },
      "source": [
        "The code below checks the state field every 5 seconds until it is no longer in the `CREATING` state."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Oleh5athmo-4"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "21.28% - {'step': 40, 'epoch': 10, 'meanLoss': 2.4871845, 'computeTime': '2023-09-20T00:23:55.255785843Z'}\n",
            "\n",
            "21.28% - {'step': 40, 'epoch': 10, 'meanLoss': 2.4871845, 'computeTime': '2023-09-20T00:23:55.255785843Z'}\n",
            "\n",
            "43.09% - {'step': 81, 'epoch': 21, 'meanLoss': 0.032220088, 'computeTime': '2023-09-20T00:24:56.302837803Z'}\n",
            "\n",
            "43.09% - {'step': 81, 'epoch': 21, 'meanLoss': 0.032220088, 'computeTime': '2023-09-20T00:24:56.302837803Z'}\n",
            "\n",
            "63.83% - {'step': 120, 'epoch': 32, 'meanLoss': 0.0030430648, 'computeTime': '2023-09-20T00:25:57.228615435Z'}\n",
            "\n",
            "63.83% - {'step': 120, 'epoch': 32, 'meanLoss': 0.0030430648, 'computeTime': '2023-09-20T00:25:57.228615435Z'}\n",
            "\n",
            "85.11% - {'step': 160, 'epoch': 42, 'meanLoss': -1.1145603e-06, 'computeTime': '2023-09-20T00:26:57.819011896Z'}\n",
            "\n",
            "100.00% - {'step': 188, 'epoch': 50, 'meanLoss': 0.00040101097, 'computeTime': '2023-09-20T00:27:40.024132813Z'}\n",
            "\n"
          ]
        }
      ],
      "source": [
        "import time\n",
        "import pprint\n",
        "\n",
        "op_json = operation.json()\n",
        "response = op_json.get('response')\n",
        "error = op_json.get('error')\n",
        "\n",
        "while response is None and error is None:\n",
        "    time.sleep(31)\n",
        "\n",
        "    operation = requests.get(\n",
        "        url = f'{base_url}/v1/{op_json[\"name\"]}',\n",
        "        headers=headers,\n",
        "    )\n",
        "\n",
        "    op_json = operation.json()\n",
        "    response = op_json.get('response')\n",
        "    error = op_json.get('error')\n",
        "\n",
        "    percent = op_json['metadata'].get('completedPercent')\n",
        "    if percent is not None:\n",
        "      print(f\"{percent:.2f}% - {op_json['metadata']['snapshots'][-1]}\")\n",
        "      print()\n",
        "\n",
        "if error is not None:\n",
        "    raise Exception(error)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sQRt4u1hIZ9a"
      },
      "source": [
        "### Run inference\n",
        "\n",
        "Once the tuning job is finished, you can use it to generate text in the same way you would use the base text model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZVVrt-yiIZ9h"
      },
      "outputs": [],
      "source": [
        "import time\n",
        "\n",
        "m = requests.post(\n",
        "    url = f'{base_url}/v1beta3/{name}:generateText',\n",
        "    headers=headers,\n",
        "    json= {\n",
        "         \"prompt\": {\n",
        "              \"text\": \"9\"\n",
        "              },\n",
        "    })"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "i-cFTviPIZ9h"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "9\n"
          ]
        }
      ],
      "source": [
        "import pprint\n",
        "print(m.json()['candidates'][0]['output'])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kI6PAEx4fN_M"
      },
      "source": [
        "The output from your model may or may not be correct. If the tuned model isn't performing up to your required standards, you can try adding more high quality examples, tweaking the hyperparameters or adding a preamble to your examples."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yRDAbJdYBXuj"
      },
      "source": [
        "## Next steps\n",
        "\n",
        "* See the [tuning quickstart with Python](tuning_quickstart_python) to start coding with the tuning service.\n",
        "* See the [tuning guidance](../guide/model_tuning_guidance) for more details on how to best tune the model for your use case."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "tuning_quickstart_rest.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
